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ABSTRACT 

This paper proposes a systemat ic method of measuring 
subjects' free recall from printed hard news stories, based on schema 
theories of cognition. Citing literature that demonstrates the role 
of text structures and text schemas in the recall of written text, 
the paper suggests incorporating these processes into the assessment 
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Toward a systematic method of measuring free recall 
from printed news 3 ernes 

Gigi Durham 

This paper proposes a systematic method of measuring subjects' free recall from printed 
hard news stories, based on schema theories of cognition. Citing literature that 
demonstrates the role of text structures and text schemas in the recall of written text, the 
author suggests incorporating these processes into the assessment of recall of news. In 
this paper, the first steps toward developing such a measure are taken; the procedure 
hinges on parsing stimulus and response passages according to a schematic news 
structure. A pilot study of the instrument's reliability and validity is included, although 
further work needs to be done to refine the instrument. 
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Toward a systematic method of measuring free recall from printed news stories 

In the past quarter-century of mass communication scholarship, a substantial body of 
research has focused on audience recall of the content of news stories. The bulk of these 
studies deal primarily with the quantification of news recall (e.g. Booth, 1970; Neuman, 
1976; Gunter, 1980, 1981; Findahl & Hoijer, 1975, 1981, 1985; Edwardson ; Grooms & 
Pringle, 1976; Edwardson, Grooms & Proudlove, 1981; Edwardson, Kent & McConnell, 
1985; Furnham & Gunter, 1989; DeFleur & Cronin, 1991; Wicks, 1992; DeFleur, 
Davenport, Cronin & DeFleur, 1992; Beentjes, Vooijs & Van der Voot, 1993; Feccorro 
& DeFleur, 1993). Remarkably, in each of these and other studies, different 
measurement instruments have been developed to assess subjects' recall of the stimulus 
news stories. Many of these measures are loosely based on standard tests of aided and 
unaided recall, also referred to as recognition and recall. While these measures have 
each been shown to be reliable within the discrete contexts of the studies, no attempt 
has been made, thus far, to develop a theoretically derived, reliable, valid and systematic 
method of accurately measuring memory for news that could be used with consistent 
results across a variety of experimental methods. 

The goal of this paper is to present the first steps in developing such a measure . 

The method presented in this paper is based on the information-processing model of 
schema theory. 

Schema Theory and the Recall of Printed Text 

Research indicates that recall is an interactive operation in which a reader brings 
his/her prior knowledge and beliefs into play while encoding new information and Ian r 
activates that knowledge for retrieval of the information (e.g. Bobrow & Norman, 1975; 
Schank & Abelson, 1977; Pearson, Hansen & Gordon, 1979; Graesser & Nakamura, 
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1982; Freebody & Anderson, 1983; Stahl & Jacobson, 1986). The theoretical position 
on which these findings are predicated is known as schema theory. 1 

Schema-based models of information processing have led to increased understanding 
of the recall of written text. Central to the development of such models is the evolution 
of the construct of the schema as a paradigm for cognition. 

The psychological concept of the schema emerged initially as a reaction to the 
traditional associationist models of memory and learning (e.g. Ebbinghaus, 1964), in 
which recall occurred simply as a response to a stimulus. The associationist model 
gradually gave way to the trace theory of mental representation, which evolved into 
schema theory as we know it today. (For a more complete account of the history of 
schema theory, see Hastie, 1981). The notion of a schema was first used in studies of 
memory and remembering and later applied in the study of reasoning, learning, language 
processing, problem solving, reading, and countless other cognitive and psychosocial 
processes. 

In general terms, a schema may be defined as a dynamic, generic mental framework 
for the hierarchical representation of knowledge. Anderson (1977) asserts, "A schema 
represents generic knowledge; that is, it represents what is believed to be generally true 
of a class of things, events, or situations" (p. 2). Graesser and Nakamura ( 1982), in an 
extensive exposition on the role of schemas in comprehension and memory, define 
schemas as "generic knowledge structures that guide the comprehended interpretations, 
inferences, expectations, and attention. A schema is generic in that it is a summary of 
the components, attributes and relationships that typically occur in specific exemplars" 
(pp. 60-61). 

In the schematic view of cognition, incoming information is encoded and stored via 
an appropriate schema or pre-existing mental knowledge structure. 

1 In this paper, the word "schema" will be pluralized as "schemas/ 1 per the style used by Mandler ( 1984, p. 
2. note 2), rather than the traditional "schemata." 
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Thus, in the context of recall of written text, schema theory is predicated upon the 
notion that knowledge is organized into dynamic knowledge structures in the brain that 
are activated during the reading process, as well as during other types of cognitive 
processing. As Adams and Collins (1977) have observed: 

The goal of schema theory is to specify the interface between the reader and the 
text — to specify how the reader's knowledge interacts with and shapes the 
information on the page and to specify how that knowledge must be organized to 
support the interaction, (p. 5) 

In relation to the processing of mass media news messages, several studies have 
shown that prior knowledge is vital to comprehension and recall of news (cf. Findahl & 
Hoijer, 1981; Findahl & Hoijer, 1985). In other words, the existence of a schema for a 
news topic or for some other aspect of a news story will improve recall of the story. As 
Findhal and Hoijer point out: 

Schema theory stresses the organization of earlier knowledge in memory in 
general or prototypical schemata, representing standard situations, events, or 
structures. Two different kinds of schemata have been proposed: one deals with 
knowledge about recurrent events and situations . . . ; the other deals with 
knowledge about the typical structure of stories .... In news comprehension and 
recall, both kinds of schemata (about recurrent events and about the structure of 
news items) are probably activated. (1985, p. 390). 

In a departure from the strictly numbers-oriented tradition of this line of research in 
mass communication, Woodali, Davis, and Sahin (1983) proposed a theoretical 
framework for memory and understanding of news based on principles of episodic 
memory and on the trace theory of memory and understanding. Another pioneer in this 
domain, Doris Graber (1988), conducted in-depth interviews with 21 subjects to study 
their schema-based strategies for processing information gained mainly from the news 
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media. These theoretical perspectives are strongly tied to the schematic model of 
information processing. 

Text Structures and the Reading Process 

In his groundbreaking book Remembering (1932), Frederic Bartiett of Cambridge 
University, England, proposed that recall of written material depends on a reader's 
schema for the structure of a written passage. Such structures, also known as text 
grammars, have been theorized to be fundamental in the organization of all written text 
(Rumelhart, 1975; Rumelhart, 1977; Mandler & Johnson, 1977; Meyer, 1977a, 1977b; 
Stein & Glenn, 1979; Marshall & Glock, 1979; Taylor, 1980; Taylor & Samuels, 1983; 
Page & Stewart, 1985; Waters & Hou, 1987; Griffith & Ripich, 1988; Cook & Mayer, 
1988;Troyer, 1992). 

News stories possess a structure cr text grammar uniquely their own. As van Dijk 
(1988a) observed, "[I]n English we may use the term c news story/ and this suggests that 
news might be a special kind of narrative. Yet, we also know that it differs from the kind 
of stories we tell in everyday conversations or in children's books or in novels. Hence, 
we must specify why and how news stories are different." (p. 1). 

That news stories follow a set structural pattern is a notion that has been intuitively 
acknowledged for many years. In the textbook Writing for Mass Comm- mcation , 
Hutchison (1986) points out that all hard news stories 2 should have a formal structure, 
beginning with a lead: 

In a good lead, the important things come first. They provide the umbrella under 
which all details of the story will fit comfortably. . . . The details usually flow from 
the lead in order of descending importance into the succeeding paragraphs. A 



2 The inverted pyramid story structurt is characteristic of hard news stories, i.e. stories that are factual 
accounts of events, usually with a time element. Soft news stories (news stories with a human interest 
focus, written in a lighter vein) or feature stories often do not follow the inverted pyramid structure. The 
present research is therefor confined to hard news stories. 

( 
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simple news story about a minor traffic accident or a minor house fire will look 
like an inverted pyramid, (p. 125). 

In another popular journalism textbook, Fedler ( 1989) describes the structure of a 
hard news story thus: 

The lead in an inverted pyramid story summarizes the topic, and each of the 
following paragraphs presents some additional information about it: names, 
descriptions, quotations, conflicting viewpoints, explanations, background data 
and so forth. Most paragraphs are self-contained units that require no further 
explanation, and only the summary of the entire story appears in the lead. News 
stories end with their least important details, (pp. 135-136). 
The structuring of hard news stories as inverted pyramids is so entrenched in the 
newswriting process that Tuchman (1978) claims that most news stories consist of 
prestmctured patterns of words into which reporters insert "factoids." 

The traditional inverted pyramid structure of news stories corresponds to the concept 
of a schematic structure or text grammar. Van Dijk (1983, 1988a, 1988b) observed that a 
news story can be viewed in terms of schematically structured discourse. Using cognitive 
models, van Dijk examined media discourse and its representation in memory, and his 
analysis of hundreds of international newspaper stories led him to postulate an 
underlying structure for printed news: 

The overall organization of news discourse reflects the importance of 
macrostructures. These will typically be expressed by titles or headlines, by initial 
or final summaries, or by leads . . . The lead, often printed in bold type . . . will 
express, in a first few sentences (which are, by definition, "thematic sentences"), 
the full macrostructure of the news discourse. Following sentences will then 
progressively specify further details of the events, with the less important ones at 
the end (with the practical consequences that these can, if necessary, be cut by 
the editor). Unlike argumentatively structured discourse, such as the scholarly 
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paper, where the important conclusion comes at the end, news in the daily press 
is organized by the principle of relevance or importance, along a dimension of 
decreasing prominence with respect to the macrostructure. (van Dijk, 19^3, 
P- 35). 

Newsom and Wollert (1988, p. 120) assert that most hard news stories have the 
following elements: 

1. The lead (the main point) 

2. Secondary points in a tie-in transition 

3. Elaboration on the main point 

4. Support for the lead 

5. Background 

6. Development of the main idea 

7. Details 

They offer two diagrammatic representations of the inverted pyramid: the 
"traditional" inverted pyramid (Figure 1-1) and the "modified" inverted pyramid (Figure 
1-2), represented on the following pages. 
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Who, what, when, where, why & how (16 to 25 words) 

One sentence connecting one element of the lead to the body 

Development of the most important WWWWWH elements of lead 

Second most important element of WWWWWH 

Further development of most important element 

Other elements 

The least important facts in the story — nothing new introduced 
LEAD 5W&H 



Tie-in 



Most important element 
Second most important element 
Other 



Least important 
element 



Figure 1-1. Newsom and Woilert's modified inverted pyramid 




LEAD: 

TIE-IN: 

BODY: 
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LEAD: 

TIE-IN: 
1ST GRAPH: 

2ND GRAPH: 

3RD GRAPH: 
4TH GRAPH: 



Major theme, could be significance of event, rather than fact 
May be two sentences 
May not include 5 W & H 

The leftovers of the 5W & H not mentioned in the lead 

Explication of the lead incident, quote, meaning or background of 
event — how something came to be 

Additional information about most important fact of lead. 
Something to give credibility or significance to lead information 

Secondary theme or supporting documentation for the lead 

Any other details, in order of significance to lead 



LEAD 



Documentation or 
explication — background or history 

Elaboration of lead 



Secondary theme 
or 

supporting facts, quotes 
Least significant details 



Figure 1-2* Newsom and Wollert's modified inverted pyramid 
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Each element in these structures roughly corresponds to the concept of a "basic node" 
in the Mandler and Johnson story grammar (Mandler &. Johnson, 1977) — the elements 
comprising the surface structure of a story, comprised of a sentence or several sentences. 
The basic nodes govern the way a story is written or rewritten after having been recalled. 

For the purposes of this study, Newsom and Wollert's modified inverted pyramid 
structure will be adopted as the typical structure for a breaking hard news story in an 
American newspaper. The modified inverted pyramid includes quotes and the possibility 
of a secondary theme in the story, which more complex news stories often contain; the 
traditional pyramid does not accommodate these elements. In this respect, the Newsom 
and Wollert pyramid is a more useful descriptor than the Fedler pyramid, which makes a 
provision for quotes but not for a second theme within a story. 

Defining Recall 

Recall is the dependent variable of interest in this study. Van Dijk (1987) observes 
that "one result of understanding a text is a representation of the meaning of the text in 
(episodic) memory" (p. 165), the direct implication being that text comprehension 
always results in the storage of information in long-term memory for later retrieval, i.e. 
text comprehension always precedes long-term recall of text information. Voss (1984) 
corroborates this notion. He writes 

While reading, the individual is assumed to interpret the text contents in terms 
of his or her own knowledge, interests, and attitude. During the interpretive 
process the individual develops a representation of the contents of the text. 
Learning is thus presumed to involve the storage of information via the development of 
the representation (p. 197) (emphasis added) 

As Belli (1986) observes, rival psychological theories have resulted in very different 
interpretations of the memory process. The mechanistic model of memory, for example, 
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views it as a passive process, whereas schema-based models regard memory as an active, 
adaptive operation. The latter position will be adopted for the purposes of this study. 

The term "recall" possesses different meanings in different contexts. In the mass 
communication literature, recall is further classified into aided and unaided recall. 
Facorro and DeFleur (1993) note, "In unaided recall, the subject reports to the 
experimenter all the facts of the stimulus material that he or she can remember. In aidea 
recall, cues are provided" (p. 592). British psychologist Martin Le Voi refers to unaided 
recall as "free recall" (1986, p. 105) and describes the process as happening in a situation 
where "the subject is free to recall any items . . . and create and use helpful cues in any 
way he or she wishes" (p. 105). Generally, unaided or free recall means the unprompted 
remembrance of information; aided recall, on the contrary, refers to the process of 
remembering information in a situation where prompts or cues are provided. 

In most studies of recall from printed news sources, research questions are generally 
framed in terms of long-term benefits to readers from the assimilation of information in 
news stories. Thus, free recall is of greater interest than aided recall. The measure 
proposed in this paper rs geared to the evaluation of free recall of news story content. 

Measuring Free Recall 

Free recall is often measured very informally, usually by means of a request to "write 
down brief descriptions" of what is recalled (Gunter, 1980) or requests for lists of facts 
remembered from stimulus stories (Wicks, 1993). 

This paper proposes a more formal method of measuring recall from a stimulus news 
passage, based on a procedure developed by Meyer (1975), adapted by Taylor (1980) and 
Taylor and Samuels (1985) and hter used by McGee (1982) for scoring recall of 
expository (nonfictional) text — a method similar to the scoring procedure followed by 
Mandler and Johnson (1977) for measuring recall of narrative (fictional) text. The 
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procedure is based on schema theory and the adherence of text passages to schematic 
text grammars. 

The measurement of recall inevitably involves 2. comparison — the recalled text is 
compared to the original stimulus passage, and a reca'i score is awarded based on the 
similarity of the two. 

In a schema-based method, the proportion of elements from the initial passage 
recalled per structural node would be calculated. The recall score would be based on the 
overall proportion of the stimulus passage recalled. The use of a text grammar — in this 
case the inverted pyramid structure — renders such a measure more rigorous in its 
capacity to account for each significant element of the story, making use of the reader's 
schematic processing of the text. In addition, the sequencing of elements is built into 
the structure of the stimulus passage as well as that of the recalled passage so that the 
measure can, if desired, also provide an assessment of subjects' sensitivity to news 
structures. 

Since van Dijk (1983, 1988a, 1988b) has established that most printed hard news 
stories conform to a specific text grammar (or "superstructure" in his terms), the first step 
in developing a rigorous measurement instrument is the parsing of a given stimulus 
passage according to that grammar. 

The stimulus passages is then administered to subjects in the context of an 
experiment; afterward, subjects are asked to write down an account of the stimulus 
passage, keeping as close to the original version as possible. 

In scoring, the recalled passage is also parsed according to the news text grammar, 
and the number of propositions remembered from each terminal node in the original text 
is noted. If less than half of an original sentence is remembered, the sentence is not 
counted as having been recalled. If about half the sentence is remembered, it is counted 
as 0.5. If more than half of the sentence is remembered, it is counted as a full sentence. 
Depending upon the intent of the study, propositions recalled outside of their nodes 
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could be scored as though they had been recalled in the proper sequence, given an 
alternate scoring scale, or discounted. 

An Example 

To test a recall measure based on this system, a sample news story was administered 
to 69 undergraduates. Subjects were given sufficient time to read the passages; the 
stimuli were then removed. 

Subjects were next asked to perform a series of distractor tasks, including filling out a 
multiple-choice questionnaire requesting demographic information and the West 
Informal Reading Inventory (West, 1978) to assess their reading level. 

The stimulus passage, taken from an Associated Press wire story, was as follows: 



PLAN GIVES BIG CATS BOOST 
Breeding program for panthers OK'd 

MIAMI (AP) — Ten Florida panthers roaming the wilds of South Florida are about to be 
chosen for a new life in captiv. .7 that may have important consequences for the survival 
of the endangered species. 

A captive-breeding program has been approved by federal and state officials to boost the 
shrinking panther population from an estimated 30 to 50 in the wild. 
"There are certain purists who say, Let them die a natural death out in the wild 1 ," said 
John Christian of the U.S. Fish and Wildlife Service. "We need to consider their views, 
but on the other hand we are charged with halting the extinction of the species and 
moving toward its recovery." 

"It's no question the population is stressed. When you get down t o a total population of 
30 to 50 animals, you get to the point of facing the brink of extinction," said Dennis 
Jordan, Fish and Wildlife's Florida panther coordinator. "We consider we have one 
viable sustaining population now in South Florida and none anywhere else." 

er|c lb 
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But some wildlife managers say the Florida panther, a type of cougar, is in nowhere near 
the danger of the California condor or the black-footed ferret when their entire 
population was rounded up for captive breeding. 

The goal of the new program is 500 breeding adults in captivity and three wild colonies 
in 20 years using high-tech methods such as radio-telemetry collars and possibly in vitro 
techniques. 

The nocturnal cats, with adults weighing 60 to 120 pounds, are smaller and darker than 
most cougars and have a unique tail crook and a cowlick in the middle of their backs. 
The panther, which hunts deer and smaller game, is a solitary hunter that needs lots of 
room — at least 40 square miles for a female and more than 200 square miles for each 
male, with little overlap. 

The panther once ranged from Louisiana to South Carolina, but widespread hunting and 
urban sprawl have pushed it into the Everglades and the undeveloped center of South 
Florida. 

Without help, experts estimate, the panther will vanish in 25 to 40 years. 

After the distractor tasks, subjects were asked to recall the stimulus passages and 
write them down using language as close to the originals as possible. Finally, a measure 
of reading comprehension of the passage was administered. This instrument (see 
Appendix A) was based nn Pearson and Johnson's description of textually explicit and 
scriptally implicit questions (Pearson & Johnson, 1978) as well as on the comprehension 
questions used in Johns' (1988) Basic Reading Inventory, a standardized informal reading 
evaluation instrument. 

Recalls were scored by parsing the subject-generated passages and recording the 
number of text elements recalled from the original from each node in the appropriate 
text grammar. An example of one such recalled passage and its scoring is provided 
below. 
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PLAN GIVES BIG CATS BOOST, 
Panther breeding OK'd 

10 Florida panthers will be taken into captivity in an effort to increase their 
population from 30 to 50. By the year 2000 they want 500 in captivity and 3 wild 
colonies. The cats require 200 square miles for males and 40 square miles for females for 
the nocturnal hunters, with little overlap. 

This isn't so bad as the California Condor or blackfooted ferret whose entire 
population was rounded up for captive breeding. 

So-and-so, from the Fish and Wildlife Service, said, "Some people say we should just 
leave them alone to die in the wild but we're also charged with bringing them back from 
extinction." 

The panther, which once roamed in an area from Louisiana to South Carolina has 
declined in population due to hunting and urban sprawl (and is now confined to the 
Everglades and the unpopulated area of central South Florida). 

The original stimulus passage was parsed as follows: 
Node I. (Lead) Ten Florida panthers roaming the wilds of Sou^h Florida are about to be 
chosen for a new life in captivity that may have important consequences for the survival 
of the endangered species. (1 sentence) 

Node 2. (Tie-in) A captive-breeding program has been approved by federal and state 
officials to boost the shrinking panther population from an estimated 30 to 50 in the 
wild. (1 sentence) 

Node 3* (Elaboration of lead) "There are certain purists who say, 'Let them die a natural 
death out in the wild'," said John Christian of the U.S. Fish and Wildlife Service. "We 
need to consider their views, but on the other hand we are charged with halting the 
extinction of the species and moving toward its recovery " (2 sentences) 
Node 4* (Support for the lead) "It's no question the population is stressed. When you 
get down to a total population of 30 to 50 animals, you get to the point of facing the 
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brink of extinction/' said Dennis Jordan, Fish and Wildlife's Florida panther coordinator. 
"We consider we have one viable sustaining population now in South Florida and none 
anywhere else." (2 sentences) 

Node 5. (Background) But some wildlife managers say the Florida panther, a type of 
cougar, is in nowhere near the danger of the California condor or the black-footed ferret 
when their entire population was rounded up for captive breeding. (1 sentence) 
Node 6. (Development of the main idea) The goal of the new program is 500 breeding 
adults in captivity and three wild colonies in 20 years using high-tech methods such as 
radio-telemetry collars and possibly in vitro techniques. (1 sentence) 
Node 7. (Details) The nocturnal cats, with adults weighing 60 to 120 pounds, are 
smaller and darker than most cougars and have a unique tail crook and a cowlick in the 
middle of their backs. 

The panther, which hunts deer and smaller game, is a solitary hunter that needs lots of 
room — at least 40 square miles for a female and more than 200 square miles for each 
male, with little overlap. 

The panther once ranged from Louisiana to South Carolina, but widespread hunting and 
urban sprawl have pushed it into the Everglades and the undeveloped center of South 
Florida. 

Without help, experts estimate, the panther will vanish in 25 to 40 years. (4 sentences) 

In the recalled story, the number of sentences remembered from each terminal node 
in the stimulus text structure is noted, per the scoring system described on page 11. Thus, 
in the recalled passage provided above, the first sentence the subject wrote was: "10 
Florida panthers will be taken into captivity in an effort to increase their population 
from 30 to 50." The subject has recalled the first part of the sentence but not the 
second, and therefore is given 0.5 for partial recall of the sentence comprising the first 
node. 

I'o 
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In the first sentence, the subject recalled part of the tie in — the goal of the program 
to boost the panther population from 30 to 50 in the wild. Again, the subject is given 
partial credit for recalling about half of the sentence in the node. (0.5) 

The subject's second sentence is "By the year 2000, they want 500 in captivity and 3 
wild colonies." This sentence is not very similar to anything in the second node, but it is 
similar to the sentence in the sixth node, the development of the main idea. The subject 
is given 0.5 for partial recall of the sixth node, because in this example sequencing is not 
being considered as part of the recall evaluation. 

The subject then wrote, "The cats require 200 square miles for males and 40 square 
miles for females for the nocturnal hunters, with little overlap." This is similar to the 
second sentence in the seventh node. Again, the subject is given 0.5 for that sentence. 

"This isn't so bad as the California Condor or blackfooted ferret whose entire 
population was rounded up for captive breeding. 1 ' The subject has almost exactly 
recalled Node 5, and is given a full point (1.0) for this node. 

So-and-so, from the Fish and Wildlife Service, said, "Some people say we should just 
leave them alone to die in the wild but we're also charged with bringing them back from 
extinction." This captures part of Node 3. The first sentence in this node was almost 
completely remembered, with the exception of the name of the quoted speaker, so tha 
subject receives 1.0 point. The subject remembered about half of the second sentence, so 
0.5 was awarded. The total score on this node is 1.5. 

The subject's final sentence was, "The panther, which once roamed in an area from 
Louisiana to South Carolina has declined in population due to hunting and urban sprawl 
(and is now confined to the Everglades and the unpopulated area of central South 
Florida)." This is a complete sentence from Node 7, so the subject receives a score of 
1 .0 for this sentence. 
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The subject's recall score is computed as follows, based on the proportion of each node 
recalled: 



Node 1. 


0.5/1 


= 0.5 


Node 2. 


0.5/1 


= 0.5 


Node 3. 


1.5/2=0.75 


Node 4. 


0/2 = 


0.0 


Node 5. 


1/1 = 


1.0 


Node 6. 


0.5/1 


= 0.5 


Node 7. 


2/4 = 


0.5 



Total score = 0.5 + 0.5 + 0.75 + 0 + 1 + 0.5 + 0.5 = 3.75/7 = 54% 
Reliability and Validity of the Measurement Instrument 

Validity 

"The validity of a measuring instrument may be defined as the extent to which 
differences in scores on it reflect true differences among individuals on the characteristics 
that we seek to measure" (Selltiz, Wrightsman & Cook, 1976, p. 169). In other words, 
the validity of a measure refers to the extent to which it is a true gauge of the construct it 
is supposed to measure. Here, the crucial question would be whether the recall 
instrument described above was an accurate measures of the subjects' memory for the 
stimulus passage. 
Content Validity 

Content validity is an estimate of the extent to which the measurement instrument is 
an adequate sample of the domain or process being measured. Content validity is often 
assessed by submitting the measurement instrument to the scrutiny of experts, who \^.ity 
that all facets of the construct or domain under investigation are represented in the 
instrument. The instruments used in this experiment possessed considerable content 
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validity because they were derived from the stimulus passage itself, were constructed 
following the methods used by earlier investigators of similar phenomena, and measured 
recall of each structural proposition within each passage. The measure thus represented 
an adequate sample of the processes under investigation. 
Construct Validity 

In this study, the dependent variable "recall" is a constaict or abstraction that 
describes a trait possessed by the subject — i.e., the ability to remember and to understand 
text. Construct validation refers to the process of estimating to what extent the 
measurement instruments measure these latent traits. 

Construct validation may be accomplished by means of examining patterns of 
correlation of a measure with other validated measures of the same trait (convergent 
validity) and by showing that the trait as measured by the instrument in question can be 
differentiated from other traits or constructs (discriminant validity). 

Curtis and Jackson (1962) have suggested that high correlations between measures 
intended to measure different but theoretically related constructs provide evidence of 
convergent validity. In this case, comprehension and recall of the stimulus passage could 
be expected to vary together: they are theoretically related but conceptually distinct 
constructs. The construct validity of the recall measure was thus estimated by measuring 
the degree of correlation between subjects* scores on a measure of comprehension and 
the measure of recall based on the same stimulus passage. The Pearson correlation 
coefficient was found to be 0.33 (N=69); this correlation was statistically significant 
(p=.0O3). 
Reliability 

Reliability of a measurement instrument refers to the steadiness of scores on the 
instrument. Reliability may be measured in terms of stability or consistency of scores 
over time; internal consistency, sometimes called homogeneity — the similarity of items 
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within a test or other instrument; and equivalence, or consistency across different forms 
of the same instrument. 

Reliability of the instrument was measured using the alternate-forms method. 
Twenty-four undergraduate students were asked to participate in the reliability study. 
The students responded to the various measures in the experimental sequence described 
above. Two days later, the experiment was repeated with the same class; however, 
students were given different stimulus passages on the second day. Thus, they were 
effectively given alternative forms of a single test. 

A coefficient of stability and equivalence was computed according to the formula: 

SD A SD B 

where A represents a subject's score on the first test (Test A) and B represents the 
subject's score on the second test (Test B), Ma represents the mean score on Test A, Mb 
represents the mean score on Test B, SD a represents the standard deviation of scores on 
Test A, and SDb represents the standard deviation of scores on Test B. (For a more 
detailed explanation of the alternative forms method of reliability assessment, see Walsh 
& Betz, 1985, pp. 50-51, and Horvath, 1985, pp. 71-85). 

The reliability coefficient computed for the recall measure was 0.64. This rather low 
coefficient could be attributed to the small size of the sample used in the reliability test. 
Selltiz, et al. (1976) point out that low reliability coefficients are not necessarily 
indicators of low validity of measurement instruments (pp. 194-197). They argue that in 
some cases, fluctuations in scores on measurement instruments from one test 
administration to another or even within a test are desirable in that tests that produce 
extremely homogenous results are not as useful for making fine discriminations among 
responses and may in fact reflect a high degree of content error. They note that the 
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assessment of reliability and validity occurs along a continuum from convergence of 
scores to divergence, depending on the correlations being computed, and that "if a 
measure can be shown to be reasonably valid ... it must ipso facto be reasonably reliable, 
since a measure with a large error component could not show such consistent 
relationships" (p. 197). 

Discussion 

As noted earlier in this paper, research on recall of printed text has been greatly 
influenced by schema theories. In the case of news stories, there is empirical evidence to 
show that hard news stones usually follow a prescribed text structure and can easily be 
parsed into fairly standard sequential elements. It would appear that the schematic 
processing of text is a primary influence in the recall of a printed news story's content, as 
it has been shown to be in the case of other types of printed text. This study represents 
an attempt to take advantage of the schematic structuring of hard news stories to devise a 
systematic method of assessing memory for news. 

This research contributes to the development of a theoretically rigorous model for 
understanding the process of recalling news and using it in the evaluation of memory for 
news content. However, this inquiry is still in its preliminary stages, and a more rigorous 
investigation of news text structures and the role of schemas in the processing of news 
messages needs to be undertaken before a standard measurement tool can be created. 
The me. surement method described herein needs to be retested and refined, particularly 
in terms of its reliability, so that it can be applied to all hard news stories across a variety 
of experimental methods. 
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The Comprehension Measure 

1) How many Florida panthers are left in the wild? 

2) What is the goal of the new captive breeding program? 

3) Briefly describe how the range of the Florida panther has changed over time. 

4) Two men were quoted in the story you just read. With what federal agency were 
they affiliated? 

5) To what other North American big cats might the Florida panther be related? 

6) Why is the Florida panther facing extinction? 

7) What is meant by the term "wildlife conservation"? 

8) Why are conservation efforts critically important in today's world? 

9) What was they key point of this story? 

10) How well do you feel you understood this story? 



A 
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